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.ABSTRACT 



Interactive computer systems are today becoming very widely accepted 

and are supplying' ever increasing services to . larger and larger numbers of 

users. However, one of the major re^naining problems to be overcome before 

such .systems can bo economically applied to such„ fields as public education 

is the need for massive amounts of high performance yet very inexpensive 

memory. This report will present one Approach to the solution of this 

problem by demonstrating the feasibility of using massive amounts of serial 

/ . - * ' ' . ' • ■ ' . 

memory which will eventually be less expensive than random access memory. 

The- use of a highly sophisticated memory controller will cause this serial 

memorv to perform at a similar level to that- of random access memory in 

this application. . - ^ / . 

. \ ■ ■ ^ ... 

. ■ ■ V - • ■ ■ 

l^his memory control approach will demonstrate the capability of a 
serial - memory having characteristic latency times of hundreds of ysecs 
(microseconds) to^ appear to the controlling computer to have ^average access 
times of tens of psecs.^ . - ' , 

To demonstrate this control structure, and to determine the applica- 
bility of ithis memory system to the PLATO computer-based education system, 
a memory controller was built and operated, a small- memory system was, built 
\and operated, and software was written to slave the memory ^cohtrol-lerL_^stem 
tV^ operation of the PLATO system. / 



TmLB OF CONTENTS. 



Chapter 



ERIC 




Page 



- ACKNOWLEDGEMENTS' . . 
ABSTRACT. - 

1 INTRODUCTION / . . < 

2 A CONTROL STRUCTURE FOR SI^^IAL SWAPPING MEt^ORIES 
2.1 Introduction » ^ 



2.2 An Overview of the AMS •Memory 



2.2.1 The AMS Memory Stpo^iag^ — v 

2.2.2 The AMS Memory Controller 



\ 



1 
6 

'6 

■p 
8 

8 

10 

2. 3 AMS Batch Execution ProcessX ..... w 1 2 

2.-4 Historical Development of ,the AMS Control Structure . . .14 
POTENTIAL SERIAL MEMORY MEDIA ^ . ....... . . • % • • — • ^ 

3.1 . Introduction . . . , • • ' • • . • • . . . . - - . . . . .. • - 

3.2 Charge Coupled Devices 

3.3 'Magnetic Bubble Memories . • ............... 

3.4- Electron Beam Memories . • . • • ' ' ' 



16 
16 
18 
15 



AUXILIARY MASS' STORAGE SYSTEM 

4.1 Introduction . . : . . • 

4.2 Conf igurational Options 



21 
21 
21 



4;2.1 Architectural Options '22 

4.2.2 Implementation Options 25 

■ ' ■' 

4.2.2.1 Controller Implementation . 25 ; 

4.2.2.2 Memory Implementation . . . . . . 27 

SySTEM-^PEEFORMANCE . . ^- :. . . • . ... • - * . • • • • • • - 35* 

5.1 lntroduct;Lon ........ ' ^^——'—^.^^ji^' • • • • • • • 35 

5.2 Controller 'PerformancQ^ .................. 

5.2^1 Performance Model Parameters . . '.35 

' • 5.2.2- Model Performance - 42^ 



/ 



Chapter ' . p^g^ 

5 SYSTEM PERFORMANCE (cont.) • . ' 

\ 5.3. Memory Performance \ ,-51 

'5.3*1' Memory Noise 51 

5.3.2 Device Failures 52 

■ ■ <t ' . ■ - ' 

'5.3.3 Memory Reliability \ . . . . .... . . 53 

6 CONCLUSION '^57 

!7 APPENDIX I qq 

7.1 Control Word Formats . . ... ....... . . ... - 59 

7.1.^1 ECS Communication Record 60 

■ 7.1.2- Control Word ... . , . , . . \ ... 60 

"^"^ 7.1.3 Status Word. . .63 

7.2 Schematic Diagrams ........... - 68 

REFERENCES ^ .... . ■ -^ A. -] -] 9 




. . r CHAPTEK -1 , . ^ . . 

" v^' INTRODUCTION , . ' 

The purpose-. of this project was to investigate a new approach tq ttie 
problem of providinq^^nta^sive' amounts of low-cost computer memory in an 
interactive computer environment which normally requires large amounts ot 
high-speed, high-cost ' random access memory. This alternative approach 
uses serial memory with a very high transfer rate in place of fast random 
access memory. The^name of this memory is the Auxiliary Mass Storage (AMS) 
system. Although several types of mem6ry devices could be used- for this 
experiment, commercially available ^emicpnductor serial shift-register 
memory is- utilised. Even though access to specific location in a serial 
memory can require significant amounts of time, the use p£ inter-leaving 
and -parallel memory control paths can provide a significant reduction. of 
the access times and permit very high transfer rates. ^ 

The interactive system used in this memory 'experiment is' the PLATO IV 
computer-based, education ' [1 >2] system, located on the eampus of the University 

■of Illinois ,- Urbana, in. the Computer-based Education Research Laboratory. - 

..... .... ^ , ..- ^ ^ . ^ ^ 

The PLATO system must process a. response in 0.1 second for each j keypress 
• . y " ■ ■ • I 

even though each* keypress is treated as an individual job for the Central^ 

• Processing Unit (CPU) . This performance requirement dictates that all of 
the necessary data arid program material be swapped fran high performance 
memory instead .of from- disks or drums ['3].- ^ . 

An elaborate memory hierarchy is necessary /for the PLATO . systemi to 
function properly, starting with a high-speed central memory of only; 65k 

' words,*'' continuing to a large mass swapping memory of two million worjds for 
the storage of working data, and finally to massive disk memory for 
storage of library materials [3], The mass swapping memory. Control 



the 
Data 



Extended Core Storage (ECS) [4] , 'represents by far the largest fraction of 



2 . • ' , ■ 

of the total cost of the computer hardware and is the ppint in the memory 

structure toward which the work' detailed in thi^s paper is directed. 

o . . ' ' ' S 

Other computer systems use other forms of mass storage where PLATO 
uses ECS but these forms are unsatisfactory for a highly interactive system 
such as PLATO^ A memory^ medium suitable- for this position in the memory 
hiefarchy* requires 'an average access time in the tens of iJsecs and a transfer 
rate in the huftdreds of millions' of bits per seconql. In contrast, magnetic ' 
disk media offer average access times of tens of thousands of ysecs aod . 
averaqe transfer rates .of only ten million: bits per second. 

Consider an interactive system servicing 1000 simultaneous users, and 
assume that each user presses keys at an average . rate ' of 0.5 key-per second, 
with each key requiring the swapping of an a:verag^ of 2000 60-bit' words of 
information (120,000 bits) between Central Memory (CM) and swapping memory. 
Operating under a memory system offering •S'O-fisecs access and 600~megabit-per- 
secpnd transfers, 500- accesses or 2.5,000 psecs and 500 transfers or 100, 000 
^ecs for a total of 125, 000 ijse'cs per "second, (one-eighth second per second)' 
would be required jiist to service the data access requirement's » Operating 
under disk parameters,. 5 million ysecs would be spent for access and 6 million 
usees for transfer. . This ia ^ total of 11 million psecs per second (eleven 
seconds/second!). 

It is not necessary that the access tim^ to data be zero to maximize. 
CPU efficiency, because in a timesharing environment the CPU can execute a 
job for one user /while the data -for the next are beiilg fetched . A typical 
job only runs a few msecs* (milliseconds) in the CPU, whereas disk transfers 
plus access take several tens of msecs, so overlapping of processing and 
access to disk is not*possible. But;if a transfer takes less than a few 
mSecs, accesse^can be totally overlapped, provided the control structure is 
appropriate. , ^ 



The basic memory module of the new mertWy uses a serial shif t-reqister 
'devicis^" which provides a 1 . ^ meqabit-per-seconKtransf rate. Using. 480 
' parallel registers , a transfer, rate of 10 million Vo-bit words per second 

is achieved, with an average aqcess^time of 400. psecs. Through the use of. ■ 
; memory p>arallelisTn and overlapping of memory accesses, the effective access 

time is reduced to about 40 ysecs and the typical performance of this new 
' memory system meets the PLATO requirements- for this level of memory. 

The concepts involved in. the organization of the AMS memory system are - 
directly applicable to several • forms of serial memory on the. technological 
horizon, inclbding' Charge Coupled Devices' (CCD) [5,6] ,. Magnetic Bubble Devices 
[7], and Electron Beam Devices [8,9]/ The P-channel, Metal Oxide Semi- 
■ conductor (MOS) technology -^was chosen for the e,xperiment principally because 
of its immediate availability. The other serial technologies mentioned above - 
promise much red^uced cost and greatly increased density and performance over 
the .current experimental version of AMS. . . 

■ In the architecture of the PLATO memory hierarchy a fundamental aspect 
is« transfer speed. The speed of a transfer and the intera:ction of that, 
transfer with the operiation of the* other parts of the system determine the' 
appropriate sizes of data swaps from one level of memory to another. From ' 
an economic standpoint it is least expensive to locate the bulk of all data _ 
siporage in the lowest cost medium that is available, but from a ^ystem per- 
formance standpoint it. is most effective to locate the bulk of the data in 
Central Memory, which provides, the highest performance . A compromise between 
these two extremes is implemen-ted in the PLATO system . Figure 1 . f . is a 
schematic block diagr^:tv of the existing memory structure of the PLATO system * . 
without utilizing an 'AMS element, and Figure 1.2 shows how AMS can be intjluded^ 
. as a fundamental system component. . In the diagram CPU means Central Processing 



12 MBlTS/StC 



TWIN CPU S 



CENTRAL MEMORY 



•600 MBITS/SEC 




EXTENDED 


CORE 


STORAGE 


(ECS) 




6 MBITS/SEC 



PLATO MEMORY WITHOUT AM S 
Figure 1.1 



12 MBITS/SEC 



Twin cpu^ • 



CENTRAL MEMORY 



■600 MBITS/SEC 




E><TENDED 


GORE 


STORAGE. 


(ECS). 



1^ 



6 MBITS 
, /SEC 

•600 MBITS/SEC 





PLATO MEMORY WITH AMS 
Figure 1.2 



11 



Onit, t>and .PPO means Peripheral Processing Ufiit. p-^VVU is a minicomputer 

... o - • 

which is primarily use'd to handle input and output. ' , 

It is important -from a*" system standpoint th 't the time spent for- data 
transfers between_.c.en^tral nif^mory and ECS be minimi2'edy because during this " 
time both of the oPUs are halted to allow all of th§ central- memory band- 
•width tp be dedicated to the transler.. This procedure degrades -CPU performanc 
Transfers between ECS arid other^; media (disk, PPUs„ or AMS) do not cause . the 
CPU t'o stop, but' they can cause a\slow-down ECS-CM 'transfers should the- 
two types of transfers exist simultaneously.- ^Disk^and PPU transfers , do npt 
offer significant conflicts b'ecause- of their low activity, but AMS transfers 
can potentially cause extreme Seg^radation of the ECS-CM transfers. For this 
reason/ a priority system was installed in iihe ECS and. AMS controllers which, 
forces the AMS to assume a lower priority to CM. T]ais s^i^stem still allows 

Idrge amounts of data to.be transferred between ' AMS /^d. ECU but avoids sig- 

■ • ■ . >' " -o ' \ ^ ■ 

nificant system degradation. * * \ " ' ■ 

^ ' ■ ' ' ' ' • ' ^ . [ . 

The result of this investigation, is the development cf a dOntrol si^ructuf 

v;hich will allow extremely 'low-cost serial memory to be ui able and which will 



provide large a^iioiants of memory with overall ^performance aomparayie 'to high- 

cost random access memory. The work has only incidentally been concerned 

* ■ ' - • ' " , • \ , " , . 

wit'- the actual serial memory utilized^to demonstrate this control structure, 

as the technologies suitable for massive serial memories have not reached 

the state of' development necessary for their imprementation . The controller 

is, howevejL-, xeadily^ a tjte^Mi^abl^-^-^^^^^^^ memory media as they 

* •■ ■ , . • . ' " „ . •■ . 

become available.. - • • 



, • .... ' -"^ v c ■ • " . CHAPTER : * 

A CONTROL STRIJCTURE FOR -SERIAL SWAPPING MEMORIES 

>« , ' " " ^ ^ • ■ 

■ " ■ •^^ ■ ■ i_ * . ■ . • ' 

' I ntroduc tion • • ^.^-^"^ • ■ ' 

Computer systems cxttempting to service large numbers of simultaneous 

uisersT a're of two basic types. On^.type is primarily, intended to provide. 

.iDatcK *job" services with the addition of small amounts of 'interactiveness 

for- the 'purpose of editing 'files and subijiit ting/ /obs.*' This -is the^-mos u ' > 

> ■ ' ■ ' ■■ ^ ■ ^" ' ' . ^ ■ r--¥ . ^ • : r- ' ■ 

common foinri.of multi-user co'mputer service, TM.e^-second tvpe. is intended to" 

. ■ . ' - ■■ ' ■ ■ ■ . / , ^ , " \ - 

-provide a high degree of . inter^^activity with j^j:s, users and thereby to 
^ef f ec jiively converse with- the users Thi^ second type of' system may not 
have the high pe^ak jierformance chairaateristics available to an Individual i 



\ 



user that tho first type offers but instead it offers 'all of the users a V\ 

l^igh level of^interaction on a continuous basis. - \ 

•, in; the normal timesharing' environment, a^ user will utilize and demand V 

interactivity while he is aditing o^r modifying -a datafile, but he will' retire. 

to a-batch job status iv/hen ^actual compilation arid execution of the job is- 

required. -In this mbde, there are two types ^bf jobs required for the computer 

t9 adequately service any user: editing f.unctaons^, such as line insertions 

and deletions, ^and batch process functions *durihg . actnal e>fecutidn^' of /the 

I. ■ ' * • 

■ > * . . . . ' ■ . ^-"^ *■ ■ ' ■ ■ ■ 

^^ilDatch- J ob . The editing functions, are . very simple and reqijire very small 

' '■ ■ " ' ■ ■ [ . . 

• ■ ■■ '■ ... ' ■ ■ ' . • , ^ 

amounts of compjater activity. The b^toh job process f unctiLonjl^'requires 
rolling the job into CM and only occasiprially rolling it oUt. for a'cces's- to 
^ the computer by^other users. Ojie important aspect of this' type of service 
is that very little swapping , of files' from some, external storage medium .is 
needed for each job. ^ ; ' ^ ■ . , • 

• * . '• 0 • ■ .' ■ 

*On-the .other hand, the more interactive* foim of -computer ;system,: an 
^dividual, "job" . in Response * to a keypress will be fairly. smaLl on the average. 



but "tcher.e are a ""large number of such, jobs per unit time. , When a CPU job 

is executed, the CPU needs access to all of the appropriate files. In order 
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to insure that the CPU does not- sit idle, those files must.be readily available.. 
. • I, * ■ • 

■ /. • 
As an. example/, consider a computer feystem used .to . teach genetic biology. ' _ 

\ ■ ■ ^\ \\ . " ' ' ^ ■ 

• Here the user is -a studen-Cand interacts with the computer by • answering - . - ^- . 

questions posed by the comput^e^* and asking questions of the computer. Every • ' 

tiip^* that a , question is answered by the student the computer has to judge 

■ that answer "in an intelligent fashion, and ■ evdry tiitie that . a question is 

?.asked by the student, the . computer is called upon to make an intelligent 

response." In^t^i'e biology example, the pomputer might need to make more tharj. 

simple matching, judgments; answers should also -»be checked .vf or misspd'lled ; 

. - * . ' - ' ■ ■ ' .; . ' . . . / ■ 

vers ions -of the correct^answer as- well. Moreover, if the student asks for , 

' ■ , ffc .' ' ■ - ' ' ■ . .' ' ' 

a new generation of a sample insect colony.,V .the compute.r would have to compute 

■ ■■ ■ ' • ■ ' ' . . '^^ ■■ , > ' ■ ' ■ . 

- the proper characteristic'" ratios according to^ known 'genetic formulas; In ' 

' <>' •■ ... "-, ^ . ^ ■ ■ . ' _ 

' • ' ■- , — 't ■ ■ ■ . ..." 

this case a keypress would require a significant amount* of CPU- processing 

with access to the appropriate programs and to a si^rtXficant amount of data., 

' In the PLATQ system, statistics 'indicate that fully half of the keys pressed 

by users are o^f the type that require significant CPU activity; ' the rest • 



merely require key echoing. " : 

In the PLATO system .the amount of time required by the CPU to service 
keypresses is' abou£ evenly distributed from.O msecs to 20 msecs , which is 

the limit allowed any particular user' during one timeslic"e. The mean is 10 

» " » ■ ■ ■ - • \ ■ , ■ • ■ ■ ' . '^ ' * ■ , ■ 

msecs. ^ Under those constraints the maximum time that can be allowed for 
.access to all of the data base that might-. be required; to service a user is " 
1.0 msecs, because if more were required , the total access time plus transfer 
• time would exceed the .CPU process time, . and the CPU would sit idle for the 
^ ,'dif Eerence. To prevent queuing problems, it is wise to make the access plus 
transfer-, time considerably shorter than the average process time. ' 

■ . ' . -.14'' 



p It is apparent .that disk-based data storage is unsatisfactory for a 

highly interactive computer system. In disk systems, the rotational access 

alone averages 8-10 msecs., and transfers are slow (typically 5 megabits per 

second). 'It is also apparent that random access data storage is far more 

than adequate because typical access times for suc^ units are 2-5 Msecs and 

dat^ transfer rates of 600 megabits per second are available. A very large 

gap exists, howeverj between these two available meSia both in performance 

and price. The memory system tha-t this thesis describes offers one suitable 

- medium to' occupy this gap. . ' 

2.2 An Overview of the AMS Memory 

The AMS .memory system was specifically designed to satis'fy the stringent 

performance requirements of' the PLATO -computer network and to provide a sv;ap- 

/ ping memory very low "in cost. This memory system consists basically of 

staiida];'d semlcoTiductor serial * sl;iift-^^gister devices , confi^gured to* allow a 

fairly sophisticated controller^ to manipulate them according to commands 

• . - . . ■ ."^^ 

issued l^y the CPU. The shift-registers provide. an average access time pf 

■ ■ ■ ■ ' • V ■ . ■ \ . ■ ■ ^ >■ 

400 psecs. and a transfer rate of« 600 megabits per'^secondj .achieved through' 

t .■..*<-■■ • , . '. • ■ . 

parallelism.- The use- of eight/independent -^ubcontrollers ca,uses the effective 



access time to approach 50 psecs. ^ • . , 

There are two fundamental elements in th'e AMS memory: the actual memory 
section and the memory conlfroller. 

■' • ■ • • ■■ ■■ 

2,2.1 -The ms Memory Storage 

Figure 2.1 illustrates the organization of the AMS storage elementSo 

. ■ ■ * ■ ' . ■ e 

The basic element utilized is a 1024 x 1 seria:l shift-register chosen to 

operate at 1.25 MHz.. "Groups of eight of these ^devices have been arranged ' 

in such a* fashio^n that a group appec.rs as" if -it^^^ere. a. single 8192 X I register 

operating at 'a maximum rate of 10 MHz. Furthermore,. 64 groups'* have been' 
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■. paralleled to form a file unit which represents the basic memory module . 

•. The basic module, called a file , .is« an 8192 x 64 serial register capable 
of reading .or writing 1 0 million 64-bit words per second. At these rates, 
and since the file is serial, the average access time to an individua^l 
storage location varies from ,0 to 819.2 ysecs. The average access time' is 
therefore 419.6 ysecs'. * 

Any hujnber of files can be installed to assemiSle a complete memory: 

...... • . ■ „ _ . £t . , 

g^roups of eight were chosen as the minimum size.' This grouping is called a 
quadrant.. For .convenience of power supply and chassis wiring , i groups of 
four quadrants are. arranged as banks . Banks can be added until th^^ total 
memory size desired is achieved. The AMS- system was designed for a ihaximum 
of 16 banks . - 

P, file isv8192 words, a quadrant 'is 65k words, a bank' is 262k words, 
and -a fully compl^eted AMS system would be^ f our megawords . A word is 64 
bits. ^ V • - - • * . . . 

^,2.2.2 The AMS Memory Controller ' ■ . ' 

'■ ~" — : ■ " ' ' ^ 

The heart of the AMS controller is ah array of eight independent cpn- 

^ ^ ^ ■ ' / '\ -. ■■ ' ^ — ^ 

gQjr l ' Gl^G. TKese unxts are caileg suJocohtrollers and have the task of . 

manipulajiing individual files for. the' purpose of data transfer into and out 

of the 'AMS memory. In addition to the subc.ontrollers , there- are several 

other units including an idle controller ' which maintains control over those 

files not under the control ^of a subcontroller , and a data channel through 

/.. ^ . • • , , . . 

which all- data and transfer parameters are communicated. 

In the PLATO systfem, the AMS memory is connected to one of the ports 

■of a high-sjDeed^^andom access swapping memory, and that memory in turn is ^ 

connected to. a high-speed port . of the CPU's central memory. * The high-speed 

random access memory, utilized is a product of Control Data Corporation: and 



is called Extended Core Storage (ECS) . This memory is characterized by a 
3.2 ysecs access time/600 megabit transfer rate, and four access ports. 
Since the CPU is the origin of all of the ms control parameters and is ' the 
unit that interprets the status information provided by the AMS controller, 

a 128-word buff er communication area is established in the random access 

... ' • J. . ■, 

swapping memory accessible by both the CPU and -AMS. The CPU plants transfer 
parameters^ ..called jobs in the communication area. Groups of eight of these, 
jobs are combined and called batches. A job contain^ the exact parameters 
for one traris:fer ('file . nxahiber, starting address within a file, starting 
transfer address in the random. access memory, and transfer length). A 

batch containing up to eight jobs is" intended to contain a list of all the 

' ' . ■ ■ ' ■ \ ' - . ■■ ' t • 

, ^ - ■ * t ■ .. J, . . 

jobs that, would be required by the CPU to service a request by an individual 
user: program files, - dat:afiles, and status information. The AMS coatroller 
treats the group' of transfers within a batch as a unit and supplies the CPU 
with status ^information relative to the entire batch as well as relating 

■ " . - • '■. . ' .s- t " ■ ■ ^ 

to the indi^'idual transfers. In this manner,, the CPU is r&lieved of the 
burden of f iguringoj^l^jjT^^ fo^ ^ particular 

vSBT have been transferred or not. 

* In practice,, two memory access operations are required to service a 

• user: , firs;t, acquire all of the files that are needed to perform the, task 
requested, and second, after processing,, return to storage those "files which. ^ 
were modified in the process. A suitable disposition of the task of rewriting 

' some f ile's (datafiles and status) is- to attach those job's to batches relating ' 

i to subsequent users. •,:•:> .* * . . 

-U^/3er normal operating conditions > the time iTeq^ired for the AMS con- 
troller to execute the first batch posted is less than the time required for 



the CPU- to compute and pogt the Remaining batches. For this' reason, the 
CPU will not be required to wait wh\i^le . eve^ the first batch is completed. ■ 

I T he basic files . within the AMS m^mory^iia^^the following access time - 

characteristics: * . . \ . . ' ■ " 

• - . maximum access time: \ 8\9.2 Msvics . ' / 

average access time: 409\.6. psecs. /• • » ■ 

However, since the AMS controller has sev^eral subcontrollers with which ' 
*to manipulate files independently, the effective access- times encountered in 
use are. greatly reduced.' In practice, the . average access times measured as 
the, time unavailable for data transfer while waiting for a file to be properly 
positioned, is reduced to tfens- of ysecs. This significant reduction in 
access times is. primarily thG"^resnlt of 1) the ability^ to independently 
manipulate all ot the. AMS files, ^ 2) the availability of eight^'file ^manipu- 
lators, ajid 3) the nature of ., the batch posting meclaanism. 
2.3 A^1S Batch Execu tion Process 

Presented here is a descriptfo'n^t>f ' the sequence executed by the AMS 
controller in the course of respondipg^ t9'* a set^- of batches posted by the 
CPU. ' . * ' ' ' 

At the beginning of a CPU execution timeslice, the CPU takes thV 

assemblage of keys that have been inputted from various terminals and 

. . ■ 

determines what .files of data will be required "'for. each* key to' be processed. 

The CPU i;h§n posts the proper transfer information in the form of ' one batch . 

■for each user, even if partial batches are the restilt. In the mean time, 

the AMS. controller has been interrogating the communication area. -As soon 

as the CPU.has jDosted the first batch> the AMS controller begdns to execute . 

that batch. The AMS controller samples the CPU-AMS 'communication area only 
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once each 30 ysecs in order to .avo:.d jairaning . the ECS data channel .with 
■unnecessary transfers,-^-. 

■ Once the . AiMS controller has recognizLad_that a batch has been posted 
by the CPU, it proceeds to assign those jobs that are contained in that . 
batch to the eight subcontrollers . . -.K .a batch is partially filled, NOP 



(no-operation) jobs are assigned to the unused .subcontrbllers. .As. soon as 

all of the jobs assQciated with a batch have.been assigned, the AMS controller 

- ' i > ' 

begins to interrogate the next location of the communication area for the 
next batch. As a batch is posted in this location, and as individual sub- 

'controllers become completed (those' with NOPs become completed immediately) , 
parts of the next batch are assigned to the idle subcontrollers.j In the 
meantime, the AMS cohtroller posts status information in the control area 
wh^re the first batch was iipicked up, overwriting the original batch param- 
eters, Ih this fashion, the AM8 controller attempts- to keep as many of 
the subcontrollers active as possible, ■ As those subcontrollers tha't were 
Issued valid jobs become ready for transfer--, they request access to the 
data channel. When granted that channel, they transfer their assigned 
data. After transferring tiieir data^, the subcontrollers return the files to 
their original position and^ ^etac}\ themselves . At this point, they* are 
ready to accept another job. . Any uKassigned jobs -in the -second batch are 

. assigned to the subcontrollers. At the point that all of the jobs in the 
original batch have been completed> '%he AMS controller writes status in^or-- 
mation to the control area one last time^ indicating that the entire .batch 
was completed, .and proceeds to pick 'up any r^aining jobs in the second 
batch, ^^^hen all of the jobs in the second batch have been accepted by an 
AMS subcontroller, the AMS controller pr^e^eeds to the next batch. This 
sequence of processing one batch at a time and starting the next batch in 



- ■ . . • ■ • • •■ ■ . . ■ ■ ■ ■ ■ ' ^ / ■ 

those subcontrollers that are idle continues until all of the batches th^t 
the. CPU has posted have been processed. . Chapter 5 discusses in deta;£l the 
resultant access^ and transfer characteristics. 

2.4 Historical Development of the AMS Control Structur e . ' ' • 

.The AMS control structure was the result of a long sequence of inter- 
action between the hardware, and software groups of -the/^PLATO system. ' There 
were three distinct design stages before the design Was finalized. 

■ The result' of the first' phase was a. controller with eight subcontrollers ^ 
as in the final version, but all of the job parameters and status information 
were .communicated via Peripheral Processor Unit (PPU) channels. After some 

evaluation . and discussion with the software staff, it became apparent that 

•! -.It • ' . ■ : . 

this method of communication would be too clumsy for the CPU to handle- and - 

too slow to take advantage of the, high performance . characteristics of the 

memory system.- In. this first design the file structure , was. somewhat different 

even though the minimum file size was 8192 words, groups of eight files were - 

slaved to each other and^were not totally independent. This - resulted in a 

reduced efficiency of the eight subcontrollers as compared to the final, 

totally independent/design. ^ 

In the secojid design of 'the AMS memory system, it was decided to 

communicate all job parameters and status through the same data channel 

through which data was .to be transferred. In this version, . however , the 

control\area consisted of just 16. words, eight used for j^b parameters' and ^ 

/eight used for status. Each job was treated individually, and the CPU was 

given the task of keeping track of which job was related to which user's . 

' ' ' ' . . " 

/request. In this version, however, the files we're made totally independent 

■ ^ ■ ^ " .- . ' 

of one another, so that just one 8k word file needed to be activated at a 



time. The problem with this design was that the CPU overhead was too h,igh, 
and the CPU could not post several and . 



processing. . . - . . 

In the third and final version of the.AMS structure, the large commu- 
nication area was. addend and the iDatch orientation was implemented. As a* , 
•result of the third design cycle, a Serial ''memory control structtire specif- 
ically designed to op'ferate in the environment presented by. the PLATO system 
was developed. Since, there was ah extensive amount of interaction between, 
the software group that would use' it 'and the hardware group that would build 
it, a very useful and powerful yet feasible structure was developed. 



CHAPTER 3 

POTENTIAL SERIAL MEMORY MEDIA 
, 3.1 Int roduction • 

• . There are .hree prime contenders for the high-perf ormartce serial .mass- 
memory market. These are 1) ^emiconductor Serial Memories (MOS and, CCD), 
2> Magnetic Bubble Memories, and' 3) Electron Beam Memories. These' three . 
differ greatly in their technologies/ 

3.2 Charge Coupled Device s ' ' • , 

The Semiconductor- Se^rial Memory is the most advanced of the thre.e 
serial m^ia. Actual- production devices have already b*^en introduced into t 
the semiconductor market, and larger faster devices are -promised shortly . 
The significantly advanced state of the semiconduc tor devices is due' largely 
to the corresponding advanced state of the semiconductor random access 
memory, which has been getting very large, attention and is expected to be 
the dominant device in the high-speed ^ high-performance memory area. The 
technologies^deyeloped for random access devices have been applied also to 
the serial memory field. Of the semiconductor devices available, CCD and ' 
MOS, it appears that 'CCD has" the advantage due to, inherent simplicity. 
However, MOS has advantages because it is-more heavily used , throughout the ' 
semiconductor industry. In fact, the "first CCD devices wh&h have been intro 
duc ed a re hybrids of the .two technologies [10]. ' • ■ 

A CCD device is basically an inactive deyice ..(contains no active tran- 
sistors at the memory-cell level) . Instead, the storage of data is similar • 
to the storage of data in a; transmission delay line but" the transmission 
line is electronically generated and is often compared to a fire bucket'"' 
brigade. Charge (water) is placed in d location (bucket) at the >'eginning 
of the line and passed, on the end of the line by electrodes (men) Along 



the way/ no water is added to the bucket and since some spills. or leaks out, 

less arrives at the end of the line than started at the beginning. In the 

case of the CCD register ,. data is . represented by the absence or. presence of 

charge at the end of the line during one cycle and this information (charge 

or no charge.) is amplified and fed back into the- beginning of the line again 

so that the line and data storage are endless. . The principal advantage->pCD 

technology has over other forms of semiconductor serial memory is_ that there 

are no .active deVibes at the cell level and thus the design o£"^^ffe~actual 

cell. is very simple. A severe disadvantage exhibited- by- the semiconductor 

bucket brigade that is not seen in the fire bucket brigade is the .fact that 

charge leaks away even when there is no transfer, so that it is necessary 

that" each data element . arrive at. the en^ of the line often enouglr that no 

data -is lost.; If "the "iTne is long (requiring a small amount of -support 

■ ' " ■ . . ■- . 

circuitry), 'the data must move -quickly from- one end to the other and con- 

< ■ ■ ■-•'.{ 
sequently use a larger amount • of . power ;^ if the line is-shdrt>.^a large amount 

of support is required and the advantage of the simple ceir is lost. An 

•* , , * ■ . , . . . 

optimum compromise can be reached: a register which has to operate at a 
medium speed so^that it can be made several hundred bits long yet not* dissi 
pate an excessive amount of power. This device will' have a limited dynamid^- 
range over which it will operate. Since varying the speed of the register 
is one technique that is used in the AMS controller to optimize the access- 
^time performance, ana this technique is not applicable w^e ..a. CCD-based 
. system implemented, some modification of the controller would ije necessary 
if CCD were use.d- ffn particular, an inteirmediate Ifevel of buffering would 
need to be added to act as a "rubber band" between the CCD memgry and ECS. . 

The projected cost per bit at the device level of CCD memory in 1977 is 
around 0.01 cents per bit. Based* on this number, the system cost would be 



; ■ • ' • . ' 18 . ■ ■ 

about 0.02 cents per bit [11].* 

• ■ " -. ■ « . , ■ ' ' ■ ■ ■ • ■ 

3.3 Magnetic Bubbl e Memories 

Magnetic bubble memories were first- developed at Bell Laboratories [12] 
and are similar to CCD . devices in being passive storage- devices /• An addi- 
tional advantage of bubble, memories is that data stored in the memory may' 
— b-e-TreraTneci ror indetinitilSiHbds. of time^ithout "the application '^^werT 
To date, however, -the development of these bubble . memories ^as ^leen slowed 
by the search for a suitable substrate material and production .devices have 
been released. In addition, bubble devices which have been operated have 

\been limi ted ^to fairly low operating speeds,. Bubble 'memories have a long 
dexelopment cycle ahead of them. \ ■ . 

The basic concept of a bubble memory is that /a thin layer of magnetic " 
material containing*' ihherently strong magnetic dipoles is exposed to a 
strong cont-inuous magnetic forpe. This material s.eparates" into domains of 
oppositely polarized areas, each with its dipoles aligned at O'' or 180'' to 
thg .apjDlied field. The separation of the domains within the magnetic layer 
is stable. If electrodes are used on the su];face of ' the material to apply ' 
additional localiz'ed fields, the domains can^ be defined and moved about- the 
surface, -In particular,' electrodes can be design,^d to cause a row of these, 
domains to move ,in a line; thus'^his row becomes a serial shift-register.. 

-One mil (1/1 000 of an incH) diameter bubbles have been .generated and .manipu- 
lated and 'this siz^e of cell- would result in 10^ bits per square inch, of . . 

\ ' . 9 ■ ■ ' . ■ ' ■ • 

material or 10 bits per cubic inch/. The cronstant* field "can^ be applied by' ' 

♦ ■ . • . ■ . .■ , . ' ' ' 

using a permanent magnet, and the application' of this fielji is the only 

: " ■ ^ ' ..■ . ■ - - v^- 

requirement for data detention iii the mentory, .A variable field can be - * ' 
applied" to the entire unit if it is desired that all of the data in a 
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partiGUlaf- unit move in the same manner^ as would be the case were it ' 
used as a serial shift-register '[12]:^ ' 

Due to speed limitations and involatility, bubble memorie^^ will lik^.ly 
be future substitutes for that level of memory now occupied by moving^ head 
disks, adding a massive increase in bit-space density and thus pbrtabjility. 
This -memory- med4um--wil^^^ ef£ei:itiv-e sourcje^^^Jiigh-- . „ 

performance memory in an AMS-type system. " 
3.4 Electron Beam Memories 

The first form of random qtccess . memories used in .early computers C9h- * 
sisted of storage .tubes, called Williams 'tubes . . These catliode-ray tubes 
nsBd charge deposited^ by the electron beam as the storage medium and achieved 
rapidVandom access through positionirt^ of the beam. The advent of 'f errite 
core technology 'displaced the. electron beam me4fum. Recently., however, in 
the search for a very high density, low- cost medium for:pa|3 memory, the 
electron , beam technology has been re-investigated. and significant results ^ 
have .been achieved. The. techniques used in the latest electron- beam work- 
oare somewhat different from the earlier versions ,^ principally in the actual 

Storage medium and the nature cf the . addressing . In the later units the 
..... , _ . - ' ■ ' . - ' ' 

Storage medium is a mUlti-layered semiconductor material, and the storage 

mechanism is 'charge deposited inside the semiconductcr rather than charge ; ^ 

deposited on a surface. Data, is written into the memory by the appli^cation 

of a bias voltage to the semiconductor- material during the application of- ' . 

tne electron beam, with the resultant/ dif;fus ion and deposition of ^ the 

electron-hole pairs created by the beam. The addressing in the early 'units ^ 

was strictly random access, whereas the addressing in the later units was 

. generally serial in. na^ture because of a high-speed data flow requirement 

and the problems of positioning an ' e:|.ectron beam to within the accuracies 



Si^^le tubes 'have ibeen demonstrated in 'the laboratory containing 

, up to 30; million bits in 100 cm''. The positioning of a'blam to this level 

♦ - • * ' ■ ' "'■ . ' ' ' . 

of accuracy m -short periods of time, would be unrealizable; rather the beam 
, . is first 'defected to a general area'and then is rede aected by a second ' 

deflection system driven by an analog ramp generator. The result' is thai 
■ . data inpul: and oiatput are in blocks and serial in nature .[1-3] . 

Electron Beam Memories promise the availability of very large amounts 

• . of medium-speed memory that" is'' non-volatile. This technique will' be a major 

^ : ■ ' . . . ^ ., . ^ , , ' . , • 

competitor of CCD devices in the .marl^et of inexpensive mas? memory. Ojie 

» • . ■■/•,... 

.disadvantage of /Electron Befam Memories is that the. basic unit of memory is 
/ ' ■ ' ' • fi ^ . • ■ . ^ ■ 

,. ^v^ry large and^, therefore, defines the minimum size memory-"' system to be 

* " ' ■ , • .' ' ■ ■ . ■ i. 

r-very larger. If -tHe basic tube is- 30 million .bits .in size, and if the 'memory 

.v ^ system requires . that parallel tubes be ifsed to keep the data transfer speed 

high_, the number of .words in^ the minimum pystem will be 30 million. Charge-- 

coupled^' devices on the other hand wi^ll be' availably in smaller sizes, i 

~^"J~ceWainXy^ 

systems and .allowing- parallelism^ at a larger than word level to- ''further 
increase the transfer speed; " . * , ■ . » * . 
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■ ... CHAPTER 4 • . . • . 

. zxT<r rtH vtt.taW MASS STORAGE SYSTEM,, ' C • 

. . i . . • • * « 

4.1 .Introduction . , h ' 

tSiven a serial memory medium as a base,, is it possible to configure it . 

and its controller in such a fashion as to make it appear to the Central 

■ . ■ . . • •- ■ ' ' ' i ' . . • . 

■ : i . ' ■ ■> „ , 

Processing System' aS if it were random access in nature or at. least bonfigure 

it in such a fashion as to allow the Processing System to use it without 

V , . ■ - . - ■ / ■ ' ■ ■ 

• appreciable degradation? ^The /answer is yes if appropriate constraints are • 
placed ^on the memory, the computer system, and the nature of the computer ^ 

service to be offered. The memor^^. system- considered is a serial s&iconductor 

.. n ■ . .. 

.memory; the computer system considered is a Control 'D^ta Cyber 73 [14] 

computer with a large aiuounjt of Extended Core Storage (ECS); and the computq^r 
service offered is that offered by the PLATO computer-based education 
project. Under these conditions, this work will show that a controller can 
be designed to interface a serial mempry^'to the PLATO system and p3;ovide 
^ memory service with very little degradation compared with a system"' using 
only ECS. ' . , ' • 

\ 4.2^ Configuration' Options ' / 

In 'the design of the 'AM S system, two distinct types of design decisions 
were required:., 1) basic architectural decisions and 2). specific implemen- 
tation decisions.. The architectural decisions have" a large bearing on the 
performance of the working system and the ability of the system to satisfy 
user requirements. "The implementation 'dedis ions determine the ease witli W 
which the unit can be implemented, the problems encountered in that imple- 

? ' ■ ' ... 

mentation, and the reliability of the final system.. ' 
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4.2.1 Architectural Op tions 

' ~ ^ • 

The basic technique adopted 'in order to cause' the AMS to perform 

-satisfactorily in the PLATO environment was to increase the transfer rate 

by parallelism and to cut the access time^with multiple control units in. 

the controller. In addition, since it is not useful to have a fast memory 

system if "it is not possible to supply transfer parameters to it quickly, 

control communications were made very fast and efficient, 

' *' /' ' * ■ , 

The ECS system that the AMS is ^designed to interface into operates at 

. • ^ / * • •■ . • . ' , - 

.speed of 10 megawords per second based on a 60-bit word. In addition, 

this transfer^ needs to be synchronized at the level of eight-word records. ' 

due to the structure of ECS. Even 'if the average data transfer rate into 

ECS were only one megaword per second/ this, data would have to be transferred 

in. sets'^pt Sight -word records at 10 megawords per second. Since it was 

necessary to support this high rate on a record b^sis, it was not difficult 

to design the AMS so that is supported « a continuous 10-megaword rate. The 



^^islt:^ Or24^iT"shif t-regi only capable of operating a conservative 

1.25 megaword per second (assuming 64 parallel registers to form a word), 

and so 512 (8 x ' 64) . shif t-registerg» were, ass^ibled and operated in parallel 

% ■ ■ ■ . ^ , 5. ■ 

to supply the desired 10 megawords per second.. Figure. 4,1 details this 

multiple register configuration used to 'generate 10 million 64-bit words per 

second with read/write ^^pability. The assemblage of 512 individual" ree?is- 

ters -shown in- Figure 4.1 provides an 8192 word, 64-bit memory sectioh: the 

secti6n is called a file and is the minimum access elem^nt^ Thirty-two of ^ 

t'hese fiiles are assembled to form a 262K word bank , and the total capacity 

of the AMS is 16 banks or four million words. . . , , 
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AN AMS. 8192 'WORD, 64- BIT FILE ' ^' 

Figure ^.1 
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Multiple access control paths were used to allow the AMS. controller 
to speed up' the. effective access times of th^.data transfers. The number- 
of these mii-ltiple paths was .chosen to be eight for several reasons. In 
the PLATO system, when a keypress is received by the CPU, this keypress 
might require the access by- the CPU ,to several different data areas in 
order to process^that keypress:', student banks, lesson material, a common 
block, and so on.* Furthermore, - all of the possibly required information 
must be available in ECS before the CPU. "can begin processing this keypress.. 
It 'is, therefore, advantageous that- the AMS controller be able at least 
. to. process simultameousl^^ all of the requirements for; one keypress.. The 
number of data' blocks was estimated to average three w'ith a reasonable 
maximum of^eight. The second factor considered in the decision of how many 
access control* paths to implement was the fact that, as detailed, below, , the 
transfer- parameters communicated -through ECS require the use o'f eight-word 
records. For the se reasons , eight ,sepai^jt,e tr^sjEe^^ 
controllers were implemented. • *** ' , 

The fastest data\communication channels' available in the Cyber 70 
series' computers are th^^aths into and .out of ECS. ^.This.path, which is 
th^ same ch^^innel used to co^mmunicate data' between AMS -and ECS, was also„ 
used to communicate the parameters of the transfers that the CPU would like 
to have the AMS controller perform. The CPU establishes a 128-word communi 
cation control area to which both AMS and the CPU .have access. ^. The GPU 
plants in this /area the jobs, and AMS plants in this area the job status. 
The 128-word area is separated^into 16 sets of eight words. The AMS con- 
troller operates pi^ihcipally on one set at a time and flags the completion 
of these sets" to the CPU so that the C^U can operate on the data that the 
AMS has, transferred. The use of 16 sets allows the CPU to , assemble many 



-sete- -of—jobs -at:-one"i:dine"^^^^^ sequentially execute the sets of 

jobs without CPU interaction. The eight jobs in a set correspond to the" 
eight subQontrollers in AMS. In addition, a single PPU .channel, capable 
of a maximum transfer rate of only 1^2 megabits per second, "'was utilized, to 
communicate basic s*etup parameters to the AM§ -controller, such, as whether 
to commence operations, ^nd where the CPU has established the AMS-CPU communi- 
cation area. ' It is important that during the course of execution of jobs 
issued by the CPU no ii^formation need.be transferred vi^ this sloW PPU 
channel, as this would greatly degrade the performance of the AMS controller 
4.2.2 Implementation Qption:^ 

After the archictectural structure -of the AMS controller was defined, 

i-here still remained the -necessity to define the various, implementation 

, * -V, .: ^ ^- ' • • . • ■ . ■ ' - ■ ' ' 

options still to be determined. Among, these were choices such'»as the type 

? _ . - ■ ^ . ■ 

of logic' to use in tjie AMS controller , the method of construction of the 

controller, the physical configuration of the memory modules and control ^ 

y ■,■'"* ■ . ■ . 

modules. The decisions were made based on the previous experience of the 
designer, on^ the mechanical constraints of the construction capabilities 
of the laboratory in which the work was to. be done, and on the immediate 
availability of the. integrated circuits. . " 

4 «2.2.1 Controller Implementation . , ^ 

• ' f 

The electronics laboratory of the Computer-based Education Research 
Laboratory is experienced in the construction of medivmi size electronic equip 
ment. The equipment previously built has generally been that necessary to 
communicate with the Cyber-70 series computers via PPU channels operating 
at a maximum speed of 1^ MHz. For this reason the structural tecHniques 
used in the AMS controller were similar, to those previously employed, with 
some modifications to account for the increased operating speeds {10 MHz) 
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and complexity. In. addition, at the time of the original design of the AMS 
controller, there' were available two logic families designed for operation 
at speeds- in excess of TO MHz: Schottky Transistor-Transistor-Logic (TTL) ' 

■ r- ■ ^ . ■ • ' ■ 

[15], and Eiriitter-Coupled-Logxc (ECL) [16], The ECS is rated to operate 
much faster than. the TTL but requires more power and does not have available 
the large assortment of complex functions that the TTL family has. The 
ECS logic IS', however, ^ truly a high-speed logic family, whereas the Schottky 
•TTL is more ^ medium-speed fcimily which has been speeded up through the 
internaJj, use of . Schottky transistor technology. All of the previous exper- 
ience of both the. designer and laboratory has been with the TTL family and 
thris outweighed the advantages of ECL. ,-The TTL family was" used. 

The construction techniques used in the AMS controller are the same 
as those previously employed in large digital laboratory projects. One / 
parcels up the required circuitry into sections which are 8-12 circuits 
each and develops printed circuit boards for each of these parcels. The 
boards are\built and interconnected through: the use of a printed circuit 
edge connector vand back plane wiring. This technique provides a very high 
density of circuitry, allowing ponvenient mixing of integrated circuits and . 

.discrete components, and takes advantag^ of whatever repetition is available 
in the circuit design. .An alternative approach would be to utilize comme|:~ 

. cially available random logic panels at - the cost of some density, i-ncreased 
difficulty in handling discrete components , and loss of the advantages' of 
repetitive circuitry. There would be two significant advantages of the 
use of these logic panels: 1 ) all of the circuitry is available^to be 
probed ^at one time, whereas with individjaal cards ""many of 'the circuits are 
hidden at any particular time, and "2) t^e parameters of the logic board are 
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'characteristics and so the problems of; operating at very high "speeds are . . 

. . ' ' . ■ ' * . ■ ■ 

somewhat simplified. ^ ^ , <. . . . 

'i'igures 4. 2 ^nd •4.3 are picture's of the AMS .controller which was 
implemented. 

4.2.2.2 Memory Implementation ^ " ^ * J 

A single bank of AMS memory (262K 64-bit words) requires the installation 
of 16,384 memory devices plus approximately 1505 other integrated circuits. 
Were the construction techniques^ used in the controller extended "to the 
memoty, rpughly 1800 cards woui^i be required, and this number is totally , ; 
unreaisonabJLe. Instead of 8-12 circuits per card, the memory was. implemented 
by examining the required interconnections and choosing a single card con- 
figuration which provided the minimum prxnted circuit 'card interconnections, 
thus saving the cost and time of the back plane wiring as well as increasing 
the reliability. A card configuration of four bits of eight files was . 
chosen which required a' printed circuit with 256 memory circuits and 16 
support circuits. Including power, '88 printed circuit connections are 
required per card. Sixty-four cards are required for a complete bank. In 

addition to the memory card there are three, other card types necessary to 

^ , ■ ■ • 

implement a memory bank: a dual file. control card, of which 16 are required, 

and two bank control cards, of which one each is required. Figure 4.4 is a 

picture of the memory card and Figures 4.5 and 4,6 are pictures of the 

memory bank which^ was implemented. Figures 4.7 and 4.8 are picture^s of the 

full quarter-million word • system. \ ' 
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CHAPTER 5 

SYSTEM PERFORMANCE 
/> . «. * 

5/1 Introduction ' 



The performance of the AMS memory system will "ibe described in twQ. _ - 
sections: 1) Controller performance, and' 2) Memory performance. 
' 5.2 Controller Performance . •* 

"The PLATO system is- heavily dependent on thfe availability of a very 
highTspeed mass memory and presently uses two million words of Extended 

Core Storage (ECS) . In this memory ^re stored many** types of dat;a necessary 

. • ' .*' ■ '* ■ 

for the -prope'r- operation of the system. The position 'of an AMS-type rftemory 
as a p£^rtial supplement to the large ECS bank and as a further ej^fe^sion 
of this memory to the much larger numbers of words necessary to service 
5>Jhousands of users depends oh its ability to provide a service level compar 
able to that of the present ECS system. The tasks that will be assigned 
to this, new memory, will be similar to but increased over those presently 
assigned to the. ECS memory. The AMS controller, whose structure -determines 
the AMS system performance, was designed with the particular character- ' 
istics of the PLATO system in mind\ The -corresponding performance -levels, 
of the controller were determined by modeling the task profile of the PLATO 
system and testing- this... model in side-by-side operation of PLATO and a 
parasitic AMS driver. ' * . ' ' , 

5.2,1 Performance Model Parameters , 

The general operation of the AMS controller can be' mode led "based on 
the variai)le parameters > both those "determined by the software usage and 
the hardware constraints. The softwa^-e determines the transfer parameters 
such as transfer lengthy file, file starting' address, and ECS. addrjsss. Not 
gener^ally under software control are variables such as .AMS-CPU conflfcts, 



file access conflicts, and phase skew between .Jih§ idle address and the . 

• , . ' 'A ^ ; • 

transfer starting address. Enough' information is availat^le to the CPU for 
it- to minimize the', AM S -CPU conflicts; file access confli'cts , and. phase skew, 
bu'': by examining the software overhead r-equired to evaluate 'this information 
it became apparent that this increased overhead would result in a net loss' 
in overall performance. ' . . 

AiMS-ECS conflicts are those occurrences of simultaneous requests "into 
the ECS controller by both the CPU and AMS. A reduction of the transfer • 
rate of AMS results because AMS^has a lower priority than the' CPU. The 
occurrences of these events can be minimized by calculating the expected 
time of an AMS-ECS transfer and having the CPU' conduct tasks which only 
require CM activity. ^ ' ^ - 

File- access conflicts are the pccurrence of a request by a particular 
PdAS subcontroller to attach a file which is already attached to another . 
subcontroller . When this situation arises, the requesting subcontroJLler 
will simply wait to start its job until, the required file has been released 
by. the other subcontroller. This situation occurs only after completion 
of the assigned transfer job. The CPU can minimi^ie the occurrence of .this 
conflict by assigni^ng only one job- pertaining "to an individual file (many 
different data areas might be contained in one 8192 word file) per job' ; , 
batch (eight transfer jobs) . • . 

The .AMS controller communicates the instantaneous rotational position 
of all of the files which are not attached to an AMS subcontroller whenever-; 
it writes status information to the ECS. communication area, . These "idling" 
files are continuously rotating at a slow rate (one revolution per 100 ysecs 
to refresh the volatile memory) -\ The CPU could utilize this information to 
choose which of a po s sib if y large number of . jobs to assign choosing first 
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those which would require ' thj/^ mdnimum rotation of the required 'file prior to 
the data transfer. This would result in a g^nerajL reduction of . the averaqe 
data acces-s times.. ' • 

The implementc^tion of these ..software- "speed-up" techniques would require 
h large amount of CPU overhead on a per job basis and 'is not practical." In 

. ; - ■ ■■ ■ . 7 

addition^ the actual -performance of the controller' and AI^IS memory system^ is ^ 

• ^ " • ' 

already sufficient ^in the PLATO environment wi<thout these further optimiza- 

t4on3. ' . - • - 

The model for the AMS controller operation takes into account the 

'.following parameters: 1) Trans:fer length, 2) Se^arch length, 3) Return 

"length, 4) ECS channel access queuing and^ CPU-AMS access conflicts. The 

AMS hardware elements involved are: 1) the actual* d^ta file., 2) the eight 

ac.cess- subcontrollers , ^^3) , the single 600 megabit per second ECS "data channel, 

1 . i^ . 

- \ ' - 

and' 4) the idle file controller. - . , - , 

_ ■ \ V' • >. .• 

. Def initions : . . ■ ' 

Transfer Length: The CPU assigns a transfer length to an AMS transfer 
'job by specifying between 1 and 1024 records to be transferred (a "record is , 
eight words). The data length^will he transferred to or from ECS sequen- 
tially from the starting address, which is also specified by the CPU. ■ This 
variable is indicated as TL "(Transfer Length) 'and has an average value TL. 

Search Length*: The CPU assigns a starting address (from 0 to 1023 

^^ ■ - . 

records) wHere the AI4S subconJ:roller is to start transferring with ECS, 

fl ■ ■ ■ • " . * 

and the subcontroller must rotate its' specific file (always in the forward 

"^direction) ^from its instantaneous positiqn to this starting position. The. 

■ " i 

Search .Length is the nutnber of record locations that the subcontroller is 

required to rotate iti file prioY to transferring data, which is the difference 

of the starting address and the insta^itaneous idle^ address at the time the 



file .was attached to the a^contro-ller. This length is assumed to be a 

uniformly .distributed i^umber between 0 And 1023 and the. time required to 

move the search length is SL (Search Length) x .8 psecs. " The average of 

this length is 512 records and is indicated as SL. ^ 

Return "Length: ©n'ce- the s<ibcon tr o 11 e r has \c on ducted^ its assigned data 

transfer, ^it must return the* file -fee' the control df. the idle controller." 

It must first align' the file with all of the other files controlled by -the 

idle 'controller and then detach itself , thus .attaching the file to the idle 

, ' ■ ' ■ * ■ ' '- ' • ■ ■ . ' . ■ • 

controller. This phase of a job's execution is a function of the size df 

SL, TL,' and the number of rotational locations that the' idle controller ^ ' 

has moved since the file was detached*. If the idle controll-er ddd not rotate 

at all, the total of SL + TL -f^ RL (Return Length) would always be an anteger 

multiple (N) of 1024, which is the file length. Instead, the sum is- ' " 

102-4 + 6, where 6 = T ( time) /1 00 ■ ysecs because the. rotational Stepping 
• ^ . ■ . "■ 

rate of the idle controller is. one step $er 100 ysecs. * The total time of 

" ■■ \ ^ ) . ■ .. ■ , . ' " ' ♦ - ■ " ■ =i ^ 

SL + TL + ;rl if N = 1 is approximately 800 ysecs (1024 x . 8 ysecs) and. wo^ld 

i . ' ■ 

therefore result in a 6 df 8 -so the sum of.SL + TL + RL + 6 =*1024 + 8 = 1032 

or less than a 1% incjr^ase. 'Larger values of N (the largest possible value 

is 3) and conflictd^ for access to the ECS datci channel cause larger values 

of The worst case value is- 72 and results when a particular file is 6 

required to wait for accesfe to the ECS ' data^ channel for seven other maximum 

' • ' . ' . .• . - ■ * 

length, transfers and had a. SL = TL = RL = 1024. The worst case' value of 6 

results In a, 7% increase. Because this percentage is very small, future 
calculations will assume 6 =0. 

ECS Channel Queuing: Even though there are eight AMS subcontrollers 
simultaneously executing transfer jobs, only one channel to ECS is available 
for the transfer of data and all data must be transferred over this channel. 



This ECS -channel tiransfers at 600 megabits per second, which is a record . 

trans'fer rate 'of one record per .8 psecs. Access to the channel is granted 

• ' ■ ' . ' <^ ' 

±o- just oneAMS subcontroller at a time pn' a first come first serve basis. 

. •■ 

Once the channe"*! is granted to a subcontroller, that subcontroller maintains 

the channel until it has completed its transfer, which is a- function of the 

transf erj^:^gth. After the completion qf a transfer, the next numerical 

subcontroller requesting the channel is granted it. ^- 

When a subcontroTler has- performed the jdb of positioning its file 

properly for a data transfer, it r^que^ts acceiss to the ECS channel in order 

to conduct that transfer. If . the channel. is busy, the requesting subcontroller 

waits uutil the chan'nel'is available. / / ' 

^ AMS-CPU Access Conflicts-;- The ECS controller [17] has four ports which 

■ ■ ■ . ^ ■ .- ^* 
allow access to the ECS storage system. In the PliATO system these' ports are • 

' . ■■ •■■ / . . 

connected , to 1) the CPUs, -.2) the. Distributed Data Path (DDP) [18] which 
allows direct PPU to ECS transfers, 3) the Si^.e DoorxAdapter (SDA). [19] ' ^ 

. ,whj,ch allows the dislcs to ^communicate directly with ECS,, and 4) AMS . The 
DDP and SDA can be neglected with regard to their interaction with the AMS\ 
and CPU, because the activity in these units' is ' very small . ^ The interaction 
between the AMS^'and CP-U is significant, however, and resuWs in what is ' . 
referred to as AMS-CPU ECS Access Conflicts, to he discussed later. 

* AMS Data File: The" datafile in the AMS is an 8192 by 64-bit memory 
area of which 60 bits^ are- used as data area and a single bit is used for 
parity check.. The ^xjemaining bits are not used at this time. The datafile 

is »a dynamic data ar|ea" and qo must be periodically irefreshed^ or regenerated, 

• • ■ \ • . - . . ■ ■ ' . . 

and this function is\accomplis'hed by shifting the file at a minimum rate. 
Even those files which are not engaged in a transfer- 'job must be shifted in 
order tq' insure that whatever data is held in them remains^ intact Tl^e 



manufacturer's sjDecifications for the d^v-ice require a minimum rotation 
rate of. "o.ne shift per millisecond, and the AMS memor^^ guarantees that one 
shift is performed, every 0.1 msecs. 

, The Eight vAccess Subcontrdllers : ' The subcontr.ollers within the AMS 
svstem are the logical . units that prepare a datafile prior to a data transfer 
and then, after that transfer/* prepar,^ the file for return to the idle state- 
Those units operate totally independently once they have started to conduct 
a job. ■ They interface, with the un.^t which "hands out the jobs" as they* are 
communicated from ECS, tHe„ unit that attaches a file to a subcontroller or 

detaches a file from a subcontroller and the unit that allocates the , ECS 

■ - - . , * . • J- • • ■ ' »■ 

data channel. When the subcontrollers are operating on a file and are not 
waiting for another section of the controller such as waiting for the data 

■ ■. ■ . ■ 

channel/ they . operate at -10 megawords per second ."^ This speed is the maximum 

speed of 'file movement- \ _ _ 

■ * 

The^ECS Data Channel: The AMS-ECS data channel is .port number 4 into , - 
the Control pata ECS Memory . Controller » -Under normal conditions, each* of 
the four controller ports has equaj. access to the ECS system, and the total 
transfer capability of the four ports .combined is 600 megabits per second, 
in 60-bit words » This transfer rate ds available only if the requests made 
to the ECS controllat' are sequentially "directed to the four different bays 
of the memory (ECS) because these bays are interleaved to this level of four. 
Under the condition that one single port is ""requesting access to memory and 
is transferring blocks of data to sticcessive locations in ECS., the sequential 
request criterion is satisfied- If more than one port. is requesting access 

V ■ .-- - . _^ ' 

to memory, or if a single port repeatedly requests access to the same bay 
of memory, the sequential request criterion cannot be generally satisfied, 
and as a result, both the individual and total data transfer rates degrade- 
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. ■ ■ ^ ^ ■ " ■ . . ■ ' ./ 

In the Cyber TO series computer .systems / when, cither of the two , CPUs ^ ^ 
proceeds to condjact an ECS data transfer, hardware constraints dictate 
that no CPU activity can be simultaneously performed even in the CPU that 
i's not. executing the ECS read/write instruction. For this reason,, it is * 
vitally important' that all ECS operations required by the CPU be conducted 
at their- maximum pojssible rate to reduce the CPU dead time. If AMS activity 
is heavy, there is a large possibility that Both the CPU and AMS will request 
ECS activity at the same time, and the result would be a^ degradation in the 

performance of the CPU system. This AMS-ECS channel is different from the 

{■ ■ . «' • 

other three ports into ECS by the inclusion of a special signal which 

■ . ■ V . . " . 

indicates that the port of ECS that is assigned to the CPU system is- 

requesting ECS access; The AMS cont:^oller uses th'is new signal to reduce 

its transfer requests to a minimum when the CPU is^^^^^questing . The transfer 

cannot be totally stopped becaiise the datafiles are dynamic and require at 

least one shift per msec. This reduced rate causes only a maximum of 0.3% > 

conflicts for the CPU, though the AMS-ECS transfer rate can be significantly 

degraded. . . . . 

* Idle File Controller: All AMS files that are not under the control o£ 

a subcontroller are placed under the control q^f the Idle File Controller. 

Tljis unit performs the necessary Refreshing of the data and keeps track of 

the rotational position of each file. Since the data is in shift-registers, 

«there "is no such thing aas, absolute addressing. Instead, particular data 

bits can be in an^physical position . • The data address of the physical 

'position i^^a'fqnet^bn of how many .shifts have occurred since an arbitrary 

starting instant-. An address register is kept by the idle, controller, and 

all files are aligned with" this register. Vlhen the control of a file is . 

transferred f rpm. the idle controller to a subcontroller ,. the contents of the 



^•idle address register are also transferred to the. subobntroller . This ' 
information is used by the subcontroller to determirfe how to rotate the 
tilo. When a subcontroilor ^is f inished wi^th a fi^e and wants to. reattach 
it to the idle .controller, it rotates the file /ntil it is again aligned* 
with the idle control address reqister and then detaches. "The idl6. cont^^oller 
rotates file^ at 10,000 shifts per second.^/ 

5.2.2 Model Performance . . / . 

' ~" ' ' / , ' -'■ 

In the PLATO environment, the mosi/ indicative factor concerning the 

level of performance of the AMS controller is the time required to complete 

all of the jobs'in one- batch since/ all of the transfers related to a parti'c- 

ular user's demand would be pack^^d into one batch, and the CPU must wait or 

/ • ■ 

/ 

otherwise, engage' itself while /that batch is in process. In addition, even 

with processing overlapped with swapping, the . average time for swapping ' ■ 

in and out must ||e less th4n the average process tijrie; otherwise the CPU 

will eventually run but /6f work to do. A batch contains from one to eight 

. / j ^ . ■ • f 

jobs, with each/ job having its individual parameters. • The parameters of 

... / ■ ' - ■ ' ■ . V * 

each job and their relatiop to the parameters of the remaining jobs deter- . 

^ ' *-'''■■-'-■.' 
,mine the time reqiiired to completely execute a batch. 

For a running system, it will be shown ±hat usually only th'e execution 
time of the first batch among several will be important, because the average \ 
time required by the CPUs" to process an individual user's data-'is longer^ 
than the average time required for the AMS controller to execute the batch 
for the next user. The AMS controller will, therefore, be aheM and -con- . 
tinue to get further ahead as long as^ there are batches to do. 

The SA.mp lest batch is one containing" all null jobs or no-op's. A 
particular bit is set aside in the job-control word sent from the CPU to 
indicate a no-op job. When the AMS controller encounters a no-op job for 
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a subcbntroller, it proceeds to assign it and the subcpntroller accepts -it. 
However, the subcontroller immediately indicates that it has completed the 
assigned job and is ready for another". The time required for a subcor^troller • 
to execute a no-op job* is approximately *5 -ysecs .If all of tha subcontrollers 
have no-op jobs, the batch is 5 psecs as well.' The AMS * controller/ does 
not indicate this situation. to the CPU except once each 30 ysecs/ however, 
to avoid clogging the ECS channel with status updates.- 

The next simplest batch is one containing just one *j'ob, and this, batch 
presents a special case' of the batch, process model. When the AMS controller 
engages 'in a single job batch, no .overlapping can be employed- The batch 
executicTn time is simply the job execution time, which is: • , . 

Time (T) = SL + TL + RXi, 

• . • ■ - _ - " ■ ' _ 

(Search Transfer ;+ Return) 
• . . ■• <^ ' ... 

As stated before, the sum of : ■ 

' ' ' SL +' TL + RL - ■ ■ . • ' - 

must be 'an integer multiple o*f 1024 records if the rotation of the idle 
controller is ignored. The integer is a function' of TLand the ekact ' . 

relation "between the starting address of the transfer and the idle address . 
when the job.v/as started. If the file is positioned at a location which is 
before but not within the "area that is to be transferred, the SL will be 
that distance to the beginning of *the transfer ^area, the TL will be the 
transfer length, and .the RL will be''(1024 - TL - SL) - , In 'this case, the , 
integer is 1. If the address transferred ■ from the idle controller is within 
the transfer' length area of trie file, the file must be rotated through the 
remainder of the transfer length and around to the beginning of the transfer 



area. Then.TL is cycled through,, and at this point, more than one complete 
cycle has already been completed. The RL will be (2048 - SL* - TL) and the 



intoqer is two. In the special case, where the idle addre&t> is very close tc 
the transfer starting addresi| and the TL, is very close, to 1024, a total of 
three .revolutions miqht be • ridcessary to complete a job. The SL will be 1024,. 
the TL is 1024, and the 'rl is 1024. The integer is 3 . 

The average time to. process a single job ba-tch assuming a uniform dis- 
tribution of job lengths and no correlation between , the 'starting address 
rmd-.the idle address,' is: 



P(N=1) X819.2 + P(N=2)x2x819.2 + P (N=3 ^^3^8 1 9 , 2 ysecs 
where P,(N=x) is the probability that the number of cycles equals x. . ' 
r;^ P(N=1) is the probability that only a single rotation will be required, 

which is the probability that the idle- address falls outside of the transfer 
. field. If we assume an average transfer length of 512, we have: 
P(N=1) = TL / 1024 = 512 / 1024 = .5. 
' . ' ;P(N=2)' is the probability that two rotations of the file will be required 

to transfer and return, which is the probability that the idle address falls 
within, the transfer field. ^ . * 

P{N=^2) = (1024 -\^y / 1024 = 512 / 1024 = . .5. 

P(N=3) is'the probability that three rotations of the file will be 

• ■ . .• ■ ■ ' , 

required in.orc^er to process a' job. This * condition occurs onl^ when the 

• idle address is almost exactly the 'same as the transfer s tar tih%^^ address . 

Its probability is very small and will be neglected. ■ ^ 

Therefore, the average time to' process a single job, . assuming that tio 

other jobs are competing for the ECS channel and ^ the CPU-AMS conflicts ar,e 

negligible, is; ' 
, . , - T = .5 X 819.2 + .5-x 2 x 819.2.= 1.2.30 yseJs: / . 
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^ For non-uniform distributions of TL, the probability of a batch 
requiring 2 revolution^ is: 




and 



and ^ 

Average J*ob Time - Average "Batch 'Time = 

• . " 819.2xP(1 024)+l638.4xPl2048)o . / ^ 

Given a linear distribution from a maximum at TL' - 1024 going to zero . 

at TL = 0, P(1024) = 1/3 and P(2048) = 2/3 and Average. Batch Time = 1365 ysecs 

More typical batches encountered in normal operation of the' AMS- system 

In the PLATO environment are those which contain more than, one job. In the 

evaluation, of these more complex jobs, additional factors need to be con- 

sidered. In particular, since more than one subcontroller is operational-, 

only the access time to the first job will be of concern, and the queuing 

for the AMS -ECS channel will be dominant, ^ 

^ The overall time required to execute ,a batch is determined by the sum 

of the access time to the first job,^ total time to transfer, and the average 

time to return the last 30b to the idle cpntroller. Since it is assumed 

s that there is no correlation between the idle address- and the file starting 

addresses and that the starting addtes^s are randomly placed, the value of- 

the first term can be . calculated as a function of the number of. jobs in a 

t 

batch anja the P (TL) . /' ■ ' - • 

The. distribution of access times to the first job for a uniform distri- 
bution 'of TL can |De shown to be: 



• ' . 1023 * 

/° ^ . • 

P(FSL) = / (TL/ld24)>^n(1-TL/1024) d(TL). 

and results in an access Icnqth with an averagfe value of 1 024/ (n^l ) . 
• ^ . ' • ""^ , ' ' ' 

For non-uniform distributions of access "timf^s the distribution of • 

•*.--■ 

r'(FI^I.) is,: • . , ■ - . • ■ . 

. 5023 ' - ■ > 

P(FSL) = / F(TL)xnX(1~P (TL) ) '^L) 
• ^ . J ' ' 

In" the case: of a uniform P(TL) , the average difference between the 
access time- of the first job and the ^pcess time of the second ^job^ which 
determines the amount of data that' must be transferred in order for the 
access time to the second job to be covered up is i : . 

(.1/n)-(l/(n+1))=1/nx(n+1)).. • - . 

. / • • ■ ■ ■ 

For a batch using' all ei^ht subcontrollers > the average access to the 

first batch is 1024/9 or 1 1 3 records or 91 ysi^cs.^. The length from the 

. - ■- ' J. ■ 

access to the Tirst job to the second job is M.2 'records or 113 words. In 

^ the PLATO system> the minimum usable data parcel is greater than 400 wordS> 

and ^therefore a vpr^y high probability exists that the transfer of the. data 

from the first job will completely overlap the access to the second job. 

■ ^ , • ^ ■ ■ - ■ ■ . , ■■ ^ 

.1 ■ 

Once the first job 'has been accessed^ the ECS data channel , is- the 
time-determinant factor > and as such, the sum of all of the >transf er length 
enters irttp- the total^ batch time.- Afte;: the last, has fiiafish^d with" the 



and at this point 
/ailable for this 
cycles, - 



data .channel, it must be returned to *its.^ idle position 
the batch will be complete. There is no overlapping ai 
return length; thus, the expected value is simply 512 
The total batch time therefore is as follows: 

BT=(1024/(n+1)) + (nXTL)+5l2 cycles 
if -the minimum transfer length is 1 024/( (n+1 ) xn) . .Table 5. .1 shows expected 
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total batch time and miniinuin transfer lengths to maintain high AMS efficiency 
as a function of the n and TL. These numbers assume: that the distribution 
of transfer lengths is uniform, that the transfer starting address is un- - 
correlated to the idle addres;5>. and that. 'the 'transfer addresses are randomly 



located. 



Table- 5.1 

Batch Times and Minimum TL (records) 



1 

V . 



100 1053 1068 

200 1253 13158 ^ 

40Q 1653 ^1958 ^ 

600 2053 25-168* 

800 245^3 .3168 

rOOO 2853 » 3768- 



minTL * 170 51. 



1116 


1182 


1258 


1340* 


.425 


1516 


\ 1682' 


1858 


2040 • 


2225 


2316 


2682 


• 3058 


/3440 


3825 


3 11*6 


3682 


4258 


U840 


5425 


3916 


4682 \^ 


5458 


'6240 


7025 


"!.716 . 


5682 " ' 


6658 


^' 7640 


8625 


34 


24 


' >8 


^ ■ 14 


11 



a 



The total amount transferred is the TIj\^ n. 
The 'efficiency of the AMS controller can be seen; by calculating an 
effective access time Which is the difference .between , the time required to 
actually transfer the required data* and the total batch time divided -by 

til e number of different files transfe^^red. 

* . - . . * ■ 

Effective. Access Time (EAT) = (BT- (TL^n) ) /n 
. "wHen the transfer lengths are .subtracted > the effective access time is 

only a f,unction of- the number of * subcontrollers active. See Table 5.2,. 

' . ^ . ■• . 

Table 5.2- 

• . EAT vs . n . 

n 2 ^ 3 4 .5 6 . 7 a' 

/ ■ EAT 426 256, 179 136 109 91 78 

(cycles) « , ^ 

. eat : 341 ■ 205 143 109 87 • 73 ^. ^ 62 

(Useps) ' ■ . . 



AIT of* the above data applies orily to the first batch' of possibly 

soveral. In Chapter 2 , the procedure for handling multiple batches is 

dotiiilod. In this case, the time . tojDrocess batches affer the first i*s 

ontiroly liffiited by the. transfer time since, .even the first access and return 
* •* * ' ^- - . » ' " . ■ * 

timos are overlapped. . . ^ ' . ^• 

" ' * ^ ' " ' ^' 

.'IVo' additional factors 'affect the* overall performance of the AMS 
memory 'in the PLATO ^system: the possible saturation of the ECS* data ^ channel 
and . the reduction of the data;^transfer rate of the AMS due to its low 
priority with respect to the CPU. - ^ * \. 

r - " ' ' . «. ■ 

In working PLATO system, the percentage of .the available , transfer 
•-bandwidth into ECS used up by the CPUs is relatively small. This situation 
prevents the severe degr adding of the CPU performance s^inceTthe^'cP^ 
i^le while such transfers take pla'ce*. The amount of ECS-CPU ' transfer , time 

- V ■ . ■ > ■ \ 

measured in a system with 400 . users^ active is approximately, 1''8%. The 
extrapolation of this number to a 1 OOO-active-user situation gives an 
e.^tin^te of , 30% f or' CPU-ECS activity. Given this 30% requirement of the ' 

) ' ' - ' ■ " ^ 

CPU/ the measured facft that each user requires major CPU se^rvice pn an 
average of 'once each four seconds and an estimation of the c!ata base"' required" 
^to service ea:ch. user, the total utilization of the GOOnnegabit-per-second 
ECS channel cka be calculated. ■ It is estimaited that each PLATO user^will 
at mQ^t requii^e 1) student status information (student bank) of --500 words,' 
2) lesson material of an average of 5000 words, 3) common data blocks^ of 
1000 words-and miscellaneous data blocks, of less than lOOO'words. tn 
addition, the st^denf bank, and perhap's the common dat^ ^Dlock, must be 
v/ritten back. The total data requirement would be less than 9000 words 
e\ery four seconds. On the average only 2"00 words of lesson material w^ill 
be read into central memory. One thousand users would require 250 processes 



per. second ur 2^2^ .piecjawords per second at a^jnaximum. This data floW repre- 
sents' 1 35^ million, bits per second .or 22,5% of the ECS data chanr^el. Between 
the AMS and the CPU, 52.5% of the data channel might be required but 5J.5% 
would still remain for- peak Absorption and other, uses . 

As stated; it is estimated that 30% of the, ECS data channel will be 

. ■ • ' . ■ ' ■ : - , . 

consumed in CPU' transfers The AMS controller assumes a low priority with 

i- ■ ' , - • . "I 

respect to the CPU in order- to insure that AMS transfers do not „ interfere ' ♦ 

... a- 

with CPU transfers and thereby further degrade the CPU performance. However, 

~ • ' • • • 

this action does degrade th6' p^erformance ol: the AMS controller, because only 

* ■ ^ * : . 

70"^ of 6X)0 megabits per second are available, so the time required to 
transfer data files will be correspondingly lengthened* Since, after bh^ 
initial access tq dlata is accomplished, the time retjuired to process a*^ 
batch is limited by th^ time ^required to tra^nsfer the dat^^and not* the 
ticcfess to files, the •30%* reduction in the transfer rate wiil be readily^' ^-^ 
apparent in .the operation of AMS. In the calculations previously detailed, 
the access to first transfer and the time to return the last file to the 
idle controller a-re unchanged, but a 30% overhead must be added' to the 
transfer . time, .to most batch mixes ,, approximately 2'5% will.be added to 
the total batch time. The calculations concerning tlie ECS channel satur- 
at ion remain unchanged,' hov^ver^ i ' * • . 

Simulated batches were cpnfigured and submitted to the AMS controller 
to determine both the operation of the controller and the accuracy of the 
modeling used to evaluate the performance. Small jobs are the most diffdciilt 
for the AMS controller to handle because the overhead is constant for all 
transfer lengths and therefore is most evident for . short transfers The 
student'' banks required when a user requests CPU activity by pressing a key 
is the smallest data field that PLATO would use; thus this file was used as 



'the source of joips. A parasitic CPU program was run at a different control 
point from PI.ATO, at a higher priority than PliATO, and used the student- 
bank list supplied by *PLATO to assemble jobs for the AMS controller. File 

0 "5 . ' ■ 

len^lths .of 512 words were used. -Since' the parasitic program was operating 
at :\ c'ontrol point above PLATO, all of the time spent executing\this logic. 

detracted from the time available to PLATO. Various different loads "were 

- d ^ . ^ ^ ^ 

tri^d from slightly over 100 users to over 220 users, which resulted in a g 
variation from, five to eight jobs per poll on the average. The results 
of these tests are shown in Table 5,3^ ' 7* ' . 

* ■ 

Table 5.3"'*'^'. ■' 
* ^ Empirical Data . - 

Si ^ b 
Users^ Jobs/Eatch ms/Batcfh Meas. ms/Batch^ Calc ' 

116' 5.19V ' 1 .50 ,1 .499 

120 5.77 . 1-.-65 1 .522 > ' ' 

155 6.04 . 1 .66 ' 1 .535 • 

, 185 . ^ 7.15 / . 1.90 1.625 

. 220 7,87 ; 2.23 ' ^ 1 .934 • . ' 

ms/Batch Meas, is milliseconds per batch measured 

* ^ ms/Batch Calc,, is .milliseconds per batch, calculated 

11 ■ . ft . ■ . 

The measured values were' consistently slightly higher than 'the calcu- 

lated values, which m^iicates some degradation due to the CPU-ECS transfers. 

If reasonable Amourvts of CPU "time are assum^ci to determine the amount of 

ECS ^data channel bandwidth by the CPU, (5%iat 116 users to 10% at 220 users) 

plus an additional 10% for the ECS bandwidth required by ^he parasitic test 

program, the results in Table 5.4 are obtained. 

Table 5.4 , • ^ ' . 

Empirical Data vsV Calculated Data ' 
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Users 


Jobs/Batch ' 


ms/Batch 'M 

4 


ms/Batch C 


. Error * 


116 ^ 


"5.19 ' ■ 


• 1.50' 


1 .622 


7.5% ■ 


120 


5.77 


.1 • ^5 


1 .,659 


.5% 


155 


6.04 


1 . 66 


/I. 688 


•1.7% 


'"^ 185 


. • 7.15 


.1.90 


1.8i3 


-4.8% 


220 


7.87 


2.23 


2,195 


-1.6% 0 
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The average transfer per job is only 300 ysecs, which is short enough to 
m^ke AMS usable even for short jobs. ^ 



Overall*, the AMS con-E*oller proves itself to be . entirely . usable in 



the- "PLATO system even when operating with only short jabs and would be a 



usable addition to the memory hierarchy. 



5.3 Memory ^Performance 

■ ' ■, . ■ ^ 

Several problems were encountered in- the assembly and testing of the 
memory portion of the' aMS system. The principal problems were those nor- 
*mally . associ-ated with -the manufacture of ^any large digital syste^n: sy^tem^ 

t> ■ . ■ • • ■ ■ 

noise and component reliability.. , 
5.3.1 ^Memory Noise ^ ' . 

In a bank of AMS memory, there are. over 17,000 integrated circuits > 
located on 82 printed circuit cards." These cards* are interconnected for 



both pov/ei: and signal purppses through a system of back-plane wiring and 



printed-circuit edge connectors. ' There are 88 connections (some of which 

are paralleled for 'power supplies) , to each of ^the circuit cards. In addition 

it 'is the nature of the MOS meiQory circuits , <which by far dominated the 
ft 

number of circuits used (16,384 .per bank), that they conduct the majority 
of their current only during transition.s of > the clock pulses..- Because of 

• ' . r . ■ - ' 

■. ■ • . ■ * / 

this characteristic, the amount of current required of the power supplies" 
is vdry irregular both on a short-term basis and a medium- term basis. Ttfe. 
current required for an individual device is 1000' times as high during a 
clock transition than, bejiweeci -these transitions. The current for an mdi- 

vidual file is 20 times as high while it is in the active, state (being 

^ i > - ... 

searched, transf eriffed , or returned by a subcont roller) than when it. xs m 

an inactive state (under the control of the idle "fcontroller) . This large 

variation in /the power supply current requirements causes noise problems. • 
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In any power distribution system, some residual resistance exists between 

' ^ ^ J - - ' 

.the power supply and the load, and the variation in the current causes a 

• .corresponding' variation in the voltage drop betv/een the supply and the f 

lOvVa. -In- addition, in large power distribution systems, the inducta.lce in 

thO'power cabling becomes significant and the speed of the change of .the 

load current develops a* voltage drop determined by the inductance. In- 

\ ' • t . -, 

anticipation of the above problems ; the basic power supply distribution 

•* , <f ■ • . 

system utilized • 1/2" x l/2.'\ aluminum stock as main busses and three. #8 

■ f - ■ . \^ ' . . 

wires paralleled to interconnect chassis* to the main buss.. "rtiis power - A^ 
. network proved to be adequate for long tei:m (Current x Resistance)* regi^lar- 

tion. • ' . . ^ 

- ' . ^ 

Only, very massive . amounts of power bussing and very'low impedence 
power supplies aire capable of limiting the amount of power supply noise 
caused by the short-term variations in the power current. . In general, 
the power distribution system alone is not capable of handling* this noise, 
and additional capacitors (high frequency shunts) need 1;o be added. In 
the original design of the AMS memory modyles , more than ten 0.1 Ufd capacitors 
-were located on each circuit board, but after operation of the memory, an * 
additional 16 capacitors needed^o be added to each board to sufficiently 
reduce the power supply f\<Ase. Ojjlce a sufficient amount of capacity was 
_ available and it was properly distributed, the noise problems within the 
memory plane were overcome. ' \. 

5.3.2 Device Failures • ' ' - • . 

^ More ^han 17, 000 > integrated circuits, are assembled to construct a 
single bank of AMS. memory (262k words). Of these, one device comprises 
" the bulk of 16,384 and a second* is used in a quantity of 512. These two 
• devices also accounted for almost' all of the device failures encountered. 

ERIC . / 
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The modes of failure of the two devices were similar, and took two forms; 

immediate failure upon initial* insertion, and 2) failure after ^several 

hbmrs of, operation. Failure percentages" for the tvjo devices are tabulated 

in fa\le. 5. Si / . • . 

t. ■ " ♦ -. " • , 

Table 5.5 ■ , / 

Device Failure Statistics . 

\ , . Failure 

Device - Usage -.x /-r ^ \ 

^ . (Initial) (Later) • 

* ' . \mM5013 16,384 420 - 3%" %10 -.4% " - 




0026 512 4?' - 9% ' 5 - -1% . • 

^ - •■. . ^ ■ ^ . . . 

5-3.3 Memdry Reliability . - * . 

^• ■Once the .assembly and testing problems were overcome and the memory ^ 
could* be fully tested-, the major c6ncern was t^otal reliability of the memory 
system. The AMS mfemory was designed to operate in the "PLATO system, and 
since this system services almost 1000 users, it would be unsatisfactory, 
for any meihory element to fail except on very rare- occasions The memory 
section of .the AMS memory did not prove to be capable o^ such a high level 
of reliability. The reason for this general volatility proved to be inherent 
in the operation of the; P-channei MOS devices used. 

/.'The ybasic storage element within the P-Channel MOS dynamic shift 

register is a* simple capacitor. Figure 5 . % xs a schematic of this basic 

cell ahd shows a capacitor connected between the substrate and the gate of 

the transmission gates. This capacitor serves as the £?torage element and 

is Refreshed every cycle of the clock, .which also shifts the data of one 

ceil, (shown) into the next cell and. shifts the data from the. previous cell 

i^to this cell. The implementation of the capacitor is in the form of a 

-2/3 



'reverse-biased diode junctiion, which has a capacity proportional to V 

" • T • * ■ " 

and has a leakage current proportional to e , whene T/ xs the temperature of 
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Figure 5.1 
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the junction. The. amount of time^ that'data is . stored on • a capacj.tor is a 
function of the capacity and the temperature. The manufacturer's specifi- 
cation guarantees that the device will maintain data over a frequency range 
from 0.01 MHz to 2.5 MHz, corresponding to a cycle time c^f from. 100 ysecs 
to 0.4 psecs over an ambient temperature of 0°C to 70°C. The AMS system 

limits the operation of the devices to a range of 100 ysecs t9 'O.S ysecs. 

*<' . ■ ^ , ' . 

However, although the devices are rated to operate over this range, they „ 
will not reliably operate if the "frequency . is quickly varied from one extreme 
to ^the other. The reason for this failure is as follows. . . 

Leakage through the storage capacitbr is an . exppnential function 'of 
the* fcermperature. of the junction. At slow speeds, the temperature of the 
junction is very lov; (even tholagh the ambient temperature might be relatively 
high) because the nearby .transistors are not switching very fast and saA:u- ' ■ 
rated MOS transistors dissipate the bulk' of their power during the actual 
switching function. Since the capacitor leakage rate, is low, long periods 
between refreshes can be tolerated, as is true during low-speed operation; 
At the other extreme, during hi^h-speed operation , the ^temperature of the 
junction is quite -h,igh and the leakage i^ very hig^ as well. However , the 
cell is refreshed very often by the clock and so data ^is maintained. 
However, when the data rate is instantly changed from a high, rate to a 
low rate, the tejng^a:tih:e of the cell does not change instantly, and for a 
Short period of "time the temperature of the junction is high but the refresh 
rate is low.^ During^ thiy time the ihargins are 'greatly reduced- and an occa- 



sional bit is dropped. 

This problem is ver^^ difficult to observe carefully and as such has 
been diagnosed by observation and through discussions with engineers of the 
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integrated circuit manufacturer. No mention of any pJroblem of this sort 'is .' 
^■'ontained in the data sheets on the uses pf the device. • , 
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■ ■ : / 

* ' CHAPTER 6 • 

• - * CONCLUSIONS 

The purpose of this work was to discover and demonstrate a new method 

* 

of cdntrolling serially-organized memories in such *a^ fashion as to make 
them usable as high-performance swapping memories in interactive computer 
systems. This purpose was accomplished. 

In the coming years,* the semiconductor industry will be introducing 
higher density, . higher p'erformance memories. It ,is my belief that as the ^ 
density and speeds increase, optimum Vopera-t ion will be obtained by serial,^ 
memories, because. they are potentially simpler and because of their lower 

* interconnect requirements. If effective means have been demonstrated for 
the operation of these memories, they will be. more widely accepted, with 
the result being the availability of less expensive mass memory systems • 

Recent strides in the technologies associated with Random Access 
Memories (RAM) have reduced the cost of these memories below even the 
parts costs of the PnChaFinel MOS devices used in this experiment. This 
development is one indicator of the enormous momentum in the semiconductor 
industry. It also shows that when the semiconductor industry attacks a 
problem in force, as it did in 'finding a device to compete with ferrite 

' y ■ ■ ■■■ . 

\ 

core memories, very remarkable ad\rances can result. If a similarly intensive 

■ _ ^ 

effort were placed in^ finding a product competitive with magnetic disk 

memory, a yery inexpensive serial technology might emerge. CCD .and Electron 

• . ■ t ' • 

Beam memories might be the hint 'of the future technologies in this area. 

■ ' ■ . ^ ■ • - ^' - ' 

• . ■-»> . ■ , ■ • , ■ 

Onqe a really inexpensive device is available, "techniques. such as detailed . 

in this report can be applied to further impact the mass memory market. 

The details of this report do not totally define a mass .memory system 

in its optimum form. Several modifications- need to be added and more 



investigation needs to bo conducted. * In particblar , the data reliability 
problems that were encountered point directly to -the neecj for at least 
sinqio bit error correction being added'. The fact that the ECS data-channel 
transfer times .are the dominant factor in the performance the system in-^ 
dicates that additiona.1 ports ought to be added to the memory controller. 
The addition of auxiliary ports would allow dataf^rom several sources to 
^e simultaneously transferred into AMS» Much of the data transferred 



into AMS originates from sources other than the CPU, Disk transfers could 
be passed to AMS via these new ports. . Implementation of these ^nd other 
modif ica'tions to the AMS controller described in this report would result 
in an even more effective s.erial . memory controlle'r. " : 



CHAPTER 7^* ^ ■ 

... APPENDIX . . , 

7.1 . Control Word Formats 

Following are the word formats of the ^GO-bit words used to comrnunidate 
.job parameters from the. CPU to the AMS controller and the job status , from 
the AMS controller to the CPU,- This communication is conducted' via an ECS 
'communication area. • In. addition, the 12~bit word formats used by the super- 
visory PPU chamnel are shown. 



7.1.1 ECS COMMUNICATION ' RECORD FORMAT 



.SEEKER 


NO, 


0 


J OB /STATUS 




SEEKER 


NO. 


1 


JOB / STATUS " 




SEEKER 

r 


no: 


2 


JOB STATUS 








. ■ / 


SEEKER 


-NO. 


3 


'JOB / status/ 




SEEKER 


NO. 


4 


JOB / STATIJS 




SEEKER 


NO. 


5 


JOB / STATUS 




SEEKER 


NO. 


6 


JOB /STATUS 




SEEKER 


NO. 


7 


JOB / STATUS 



WORD 0 



WORD I 



WORD Z 



WORD i 



WORD 4 



WORD 5 



WORD 6 



WORD 7 



{ 16 SUCH RECORDS ARE IN ECS ) 
Figure 7J 



7.1.2 CONTROL WORD FORMAT 



11 10 


9 








0 


m. 


TRANSFER LENGTH (RECORDS) 


23 22 


21 








iZ 


m 


* FILE STARTING ADDRESS (RECORDS) 


35 


33 


32 






24 


m 




AMS FILE NUMBER 


47 










36 


ECS 


STARTING ADDRESS (BITS 14-3) 


59 56 


57 


56 


55 


54 


53 4i9 


m 


0 


APE 


NOP 


WRT 


eCS ADD (BITS 15-20) 



AMS 60- BIT TRANSFER CONTROL WORD 
Figure 7.2 
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Explanafcibn of parameter field designations used in Figure, 7. 2: 
Transfer Length: ^Th'e number (10 bits) is the number of records (sets of 
eigh't Words) to be transferred. • / 



Rile Starting. Address : This address f 1 0 bits) is. the address of the firsi: 

•-record within the"" designated file that is to Be .transferred. 

*AMS File Number: This number (9 bits) is the number of the actual AMS file 

' . ■ ' , ■ ^ I ^ 

that is. to be the source or sink"^fo.r the transferred data. . 

ECS Starting Address; This field (18 bits) is the address of the first 

^ • «« * < ' 

c-record within ECS that is: to be the sink or source of the 

data transfe^rred from AMS. All data is wriyben or read from 

consecutive locations aflfe^r this one. 



APE: Abgrt on parity *err,or : Jhis flag when set causes the AMS controller 
to abort a data transfer immediately if a .read parity error 
. i-s' discovered. This fl^g applies only to read AMS operations 
because on an ECS read, the ECS controller automatically 
I performs this operation. 

*NOP: No-Operation: A control word received by the? AMS Controller with this 
bit cleared (set to "0*') is interpreted identicfally as if it 
were all zeros. This- provides the^' AMS with a "pass" or 
"fill" instruction. 

'WRT: Write: An AMS control word with this bit cleared will cause an AMS 
read operation (AJVIS to ECS data transfer). If . this bit is 
set, an ECS read will result (ECS to AMS data transfer). 
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All words read by the AMS controller with bit 57 cleared are interpreted as 

I'ransfer Control WoirdS.' The ^AMS controller, sets bit 57 to a "1" when it 
wrUtes AMS Transfer Status words to ECS. ^ 



■ v 



/ 
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7.1.3 STATUS WORD FORMAT 











* 




1 


II ■ ■ 6 


'■5 "\ 




3 


2 


1 




— ^ 

, WORD W/ERROR (BITS 0-5)/? 

* • * / ' 


RE 

1 


■ s\ 


R - 


•■ A 


JC 


B 



23 ?2 


21 


20 


19 


18 ' ,. ' ,„ 12 


CYCLES 


m 


E/0 


CA 


WORD W/ERROR (BITS 6 r 12) 















0 








35' 






26 


25 


24 


- IDLE FILE ADDRESS . 


AC 






59 58 57 56 55 54 53 52 51 ' 50 49 - -48 



BD- 



BE 




R^ 



,^6 



R. 



R: 



Ri 



AMS 60 - BIT STAJUS WORD 
figure 7.3 



V 



70 



Kscp.la'iiatiion.of terms and fields used in Figure 7.3: 

E: Hiuuuj-ed: This bit is 0 if bH^O seeker correspondincj to this word number 

is prestMitly qnqaqed. in the execution of a job. Otherwise 

. . it IS a 1 . 
* ^ 

- Job Conflict: This bit* indicates whether job conflict exists for * 

s ■ ^ . . . 

this SG^-kei;. A job conflict is the coitdition whereby new 
job parameters |Lre being supplied to a seeker which is already 
busy. This is not' an error condition but rather ^indicates 
that ^ the seeker in question has a new job to start when it 
completes the one it is presently performing. 

A: Abort: This, bit indicates that this seeker has aborted its job pre- 
maturely due to either a read error in AMS, or an abort 

^ ■ ■ - ■ 

■ indication from the ECS cpntrollet. 

* . tf . ' , ^ , ' ' ^ • 

R: . Return: This bit indicates that this seeker is pjresently in its. return 
• - * phase of a, job execution. . » ' , ■ 

S; Search (Seek): """-is bit indicates- that this seeker is presently in its 
' search pr seek phase of a job execution. 



PE: Parity. Error: This bit i^ndicates that a parity^ error has been de-B^pted 
^ ^ither by the ECS controller on an ECS read ' or by the AMS 

controllex on an AJMS read, " 

Word with Error: This fiBld (13 bits) indicates the address of the word that 
flagged the parity error if PE is set during an AMS read v 
, opera tioh. On an i:_S r.ead, only the top 10 bits are nieaning- 
- . " ful and indicate the AMS address pr,esen% wh^ the parity 



•. . ' • ■ 65 : - 

error was 'sensed. The exact ECS. address can be determined . 
from this ^nd the starting ECS address, 

CA:.- Control Abort: This, bit indicates that an abort ^was issued by -the ECS 
controller during the last control read/write operation. 

E/0: Eveh'/Odd: The . AMS controller writes the least significant bit of the 

■ ' '^^ , ■ 

batch number that an individual subcontrpller obtained its 

, job from in this location. When cP ^tatus word is written by 

the AMS controller J any individual subcontroller might be • . - . 

either processing a job from that batch or from the next 

batch. By examining the batch number and the E/0 bit, the 

CPU can/determine which situation exists at that moment. 

This /^lag bit is intende'd as a maintenance and statistics- . 

. • gat'Kering aid and would- not normally be examined by the CPU. _ 

• / • ■ . . ■ V . . . 

-Cycle: The nunlber of total memory cycles (records dj.vided by 1024) that, a 
. y ' . " • ' ' 

/ seeker requires to complete a job (search, transfer, return) 

is indicated here. The seeker must^always rotate a file an 

integer numbaj^oiL-times-^fxom^t^^ 

tive phase of the file must be the same (to line up with the 

idle address) at the end of job as- if was at the beginning of 

that job. This information can be used in statistical studies 

■ of performance. 

AC: Access Conflict: The setting of this bit indicates that this . seeker > 
' . 'has attempted to attach to a file that was already under the 

cQntrol of another seeker and thereby not attachable. The 
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seeker will wait until the file isr detached by the other 
seeKsr and proceed from there. • " • 

o File Ad^dr^ss: The -exact re*lative rotational position of those files 
that- are under'' the control of the Idle File Controller is* 
• • . vindicated here. The CPU can use this information to minimize 
the -Search cycle if it assigns those jolps that it has to 
assign that would require the minimum rotational shift prior 
to transfer. Use of this feature would significantly reduce 
the access time to data but has no (or little) effect on the 
total job cycle. 

■ ■ ■ , ■ ' ' 
P^: Primary^-Primary^ • These bits (the subscript corresponds to seeker 

■ numbers) indicate whether a seeker is presently .conducting 

a* primary "job (the E/O bit Is the " same as "the least? signifi- 

cant bit of the ECS address that the control word was read 

from) or a secondary job (the' E/O bit is different from* the 

ECS address). Note: The. ECS address in "question is the ' 

.record address only as the word address pertains to the 

* • ^ . ■* — ■ * 

individual seekers only. * * " 

Batch Error:'- A " V in this bit location T.liMca^es~~t"hat" the entire batch 
has been completed but that there was an error o The remainder 
of the status record will have to be examined to determine 
the nature of that error. If a seeker encolap^ers an error 
in the- execution of a joi^ it is not allowed to look ahead 
to ;^he next batch for a new job, as to do so would destroy ^ 
the error status information resulting from the error. 
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BD: Batch' Done' (without "error) : When this, bit is 'set,- the entire batch 
has been completed and therefore the CPU fean depend on the 
.data that was transferred* The CPU would typically look ' 
only at this b'it to determine if a job has been completed. 
Once this bit 'has been set by the -^S controller, the CPU is 

. free to rewrite the status information with new control 

^- . information . * ' ' " . t 

• ' Bit 57- is always written to a 1 in an AMS status word.. ' " 
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1.2 Schematic Diagrams 



Following arc copies of the schematic diagrams of the AMS controller 

** • ■ 

and momory^ system. < . . . 
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Figure 7.4 
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Figure 7.6 
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Figure 7.8 
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